Application programming interfaces for gesture operations

ABSTRACT

At least certain embodiments of the present disclosure include an environment with user interface software interacting with a software application to provide gesture operations for a display of a device. A method for operating through an application programming interface (API) in this environment includes transferring a scaling transform call. The gesture operations include performing a scaling transform such as a zoom in or zoom out in response to a user input having two or more input points. The gesture operations also include performing a rotation transform to rotate an image or view in response to a user input having two or more input points.

This application is a continuation of co-pending U.S. application Ser.No. 13/221,836 filed on Aug. 30, 2011, which is a continuation ofco-pending U.S. application Ser. No. 11/620,727 filed on Jan. 7, 2007.

FIELD OF THE DISCLOSURE

This disclosure relates to application programming interfaces thatprovide gesture operations.

COMPUTER PROGRAM LISTING

A portion of the disclosure of this patent document contains materialwhich is subject to copyright protection. The copyright owner has noobjection to facsimile reproduction by anyone of the patent document orthe patent disclosure, as it appears in the Patent & Trademark Officepatent file or records, but otherwise reserves all copyright rightswhatsoever.

Applicant has submitted herewith Computer Program Listings which areincluded as Appendix A, attached.

BACKGROUND OF THE DISCLOSURE

An API is a source code interface that a computer system or programlibrary provides in order to support requests for services from asoftware application. An API is specified in terms of a programminglanguage that can be interpretative or compiled when an application isbuilt, rather than an explicit low level description of how data is laidout in memory. The software that provides the functionality described byan API is said to be an implementation of the API.

Various devices such as electronic devices, computing systems, portabledevices, and handheld devices have software applications. The APIinterfaces between the software applications and user interface softwareto provide a user of the device with certain features and operations. Auser may desire certain operations such as scrolling, selecting,gesturing, and animating operations for a display of the device.

Scrolling is the act of sliding a directional (e.g., horizontal orvertical) presentation of content, such as text, drawings, or images,across a screen or display window. In a typical graphical userinterface, scrolling is done with the help of a scrollbar or usingkeyboard shortcuts, often the arrow keys. Gesturing is a type of userinput with two or more input points. Animating operations includechanging content within a given time period.

The various types of devices may have a limited display size, userinterface, software, API interface and/or processing capability whichlimits the ease of use of the devices. User interfaces of devicesimplement APIs in order to provide requested functionality and features.These user interfaces can have difficulty interpreting the various typesof user inputs and providing the intended functionality associated withthe user inputs.

SUMMARY OF THE DESCRIPTION

At least certain embodiments of the present disclosure include one ormore application programming interfaces in an environment with userinterface software interacting with a software application. Variousfunction calls or messages are transferred via the applicationprogramming interfaces between the user interface software and softwareapplications. Example application programming interfaces transferfunction calls to implement scrolling, gesturing, and animatingoperations for a device.

At least certain embodiments of the present disclosure include anenvironment with user interface software interacting with a softwareapplication. A method for operating through an application programminginterface (API) in this environment includes transferring a set bouncecall. The method further includes setting at least one of maximum andminimum bounce values. The set bounce call causes a bounce of a scrolledregion in an opposite direction of a scroll based on a region past anedge of the scrolled region being visible in a display region at the endof the scroll.

At least certain embodiments of the present disclosure include anenvironment with user interface software interacting with a softwareapplication. A method for operating through an application programminginterface (API) in this environment includes transferring a rubberbandcall. Rubberbanding a scrolled region within a display region occurs bya predetermined maximum displacement when the scrolled region exceeds adisplay edge. The method further includes transferring an edgerubberband call to set displacement values for at least one edge of thedisplay (e.g., top and bottom edges, left and right edges).

At least certain embodiments of the present disclosure include anenvironment with user interface software interacting with a softwareapplication to provide gesture operations for a display of a device. Amethod for operating through an application programming interface (API)in this environment includes transferring a scaling transform call. Thegesture operations include performing a scaling transform such as a zoomin or zoom out in response to a user input having two or more inputpoints. The gesture operations also include performing a rotationtransform to rotate an image or view in response to a user input havingtwo or more input points.

At least certain embodiments of the present disclosure include a methodfor performing animations for a display of a device. The method includesstarting at least one animation. The method further includes determiningthe progress of each animation. The method further includes completingeach animation based on a single timer. The single timer can be based ona redraw interval of the display hardware.

Various devices which perform one or more of the foregoing methods andmachine readable media which, when executed by a processing system,cause the processing system to perform these methods, are alsodescribed.

Other methods, devices and machine readable media are also described.

BRIEF DESCRIPTION OF THE DRAWINGS

The disclosure is described by way of example with reference to theaccompanying drawings, wherein:

FIG. 1 is flow chart of a method for responding to a user input of adata processing device;

FIG. 2 illustrates details of an application programming interface inflow chart form according to certain teachings of the presentdisclosure;

FIG. 3 illustrates details of an application programming interface inflow chart form according to certain teachings of the presentdisclosure;

FIG. 4 is a schematic diagram illustrating an embodiment of userinterface of a portable electronic device 400 having a touch-sensitivedisplay 408;

FIG. 5A-5C illustrate at least some embodiments of user interface of aportable electronic device 400 having a touch-sensitive display;

FIG. 6A-6D illustrate the scrolling of a list of items to a terminus ofthe list, at which point an area beyond the terminus is displayed andthe list is then scrolled in an opposite direction until the area beyondthe terminus is no longer displayed, in accordance with someembodiments;

FIG. 7 illustrates details of an application programming interface inflow chart form according to certain teachings of the presentdisclosure;

FIG. 8 illustrates first and second scroll angles for locking a scrollof a display of a device in a horizontal or vertical direction accordingto certain teachings of the present disclosure;

FIG. 9 illustrates details of an application programming interface inflow chart form according to certain teachings of the presentdisclosure;

FIG. 10 illustrates details of an application programming interface inflow chart form according to certain teachings of the presentdisclosure;

FIG. 11 illustrates details of an application programming interface inflow chart form according to certain teachings of the presentdisclosure;

FIG. 12 illustrates details of an application programming interface inflow chart form according to certain teachings of the presentdisclosure;

FIG. 13 illustrates details of an application programming interface inflow chart form according to certain teachings of the presentdisclosure;

FIG. 14 illustrates details of an application programming interface inflow chart form according to certain teachings of the presentdisclosure;

FIG. 15 illustrates a display of a device having a scaling transform ofa view;

FIGS. 16A and 16B illustrate a display of a device with a view having afirst and a second scaling factor;

FIG. 16C illustrates changing a view from a scale factor of 2× to ascale factor of 1× in at least some embodiments of the presentdisclosure;

FIG. 17 illustrates details of an application programming interface inflow chart form according to certain teachings of the presentdisclosure;

FIG. 18 illustrates details of an application programming interface inflow chart form according to certain teachings of the presentdisclosure;

FIG. 19 is flow chart of a method for animating views displayed on adisplay of a device;

FIG. 20 is flow chart of a method for animating views displayed on adisplay of a device;

FIG. 21 illustrates details of an application programming interface inflow chart form according to certain teachings of the presentdisclosure;

FIGS. 22A and 22B illustrate synchronizing the resizing of windows of adisplay of a device;

FIG. 23 illustrates a method for switching ownership of a view of anapplication displayed on a display of a data processing device;

FIG. 24 illustrates a method for memory management of a view of anapplication displayed on a display of a device;

FIGS. 25A and 25B illustrate a data structure having a hierarchy oflayers with a layer being associated with a view;

FIG. 26 illustrates a method for compositing media and non-media contentof user interface for display on a device;

FIG. 27 illustrates a data structure or layer tree having a hierarchy oflayers;

FIG. 28 is a perspective view of a device in accordance with oneembodiment of the present disclosure;

FIG. 29 is a perspective view of a device in accordance with oneembodiment of the present disclosure;

FIGS. 30A and 30B are perspective views of a wireless device inaccordance with one embodiment of the present disclosure;

FIG. 31 is a block diagram of a system in which embodiments of thepresent disclosure can be implemented;

FIG. 32 shows another example of a device in accordance with oneembodiment of the present disclosure;

FIG. 33A is a perspective view of a device in a first configuration(e.g. in a laptop configuration) in accordance with one embodiment ofthe present disclosure;

FIG. 33B is a perspective view of the device of FIG. 33A in a secondconfiguration (e.g. a transition configuration) in accordance with oneembodiment of the present disclosure;

FIG. 33C is a perspective view of the device of FIG. 33A in a thirdconfiguration (e.g., a tablet configuration) in accordance with oneembodiment of the present disclosure; and

FIG. 34 illustrates details of an application programming interface inflow chart form according to certain teachings of the presentdisclosure.

DETAILED DESCRIPTION

Various embodiments and aspects of the disclosure will be described withreference to details discussed below, and the accompanying drawings willillustrate the various embodiments. The following description anddrawings are illustrative of the disclosure and are not to be construedas limiting the disclosure. Numerous specific details are described toprovide a through understanding of various embodiments of the presentdisclosure. However, in certain instances, well-known or conventionaldetails are not described in order to provide a concise discussion ofembodiments of the present disclosure.

Some portions of the detailed descriptions which follow are presented interms of algorithms which include operations on data stored within acomputer memory. An algorithm is generally a self-consistent sequence ofoperations leading to a desired result. The operations typically requireor involve physical manipulations of physical quantities. Usually,though not necessarily, these quantities take the form of electrical ormagnetic signals capable of being stored, transferred, combined,compared, and otherwise manipulated. It has proven convenient at times,principally for reasons of common usage, to refer to these signals asbits, values, elements, symbols, characters, terms, numbers, or thelike.

It should be borne in mind, however, that all of these and similar termsare to be associated with the appropriate physical quantities and aremerely convenient labels applied to these quantities. Unlessspecifically stated otherwise as apparent from the following discussion,it is appreciated that throughout the description, discussions utilizingterms such as “processing” or “computing” or “calculating” or“determining” or “displaying” or the like, can refer to the action andprocesses of a data processing system, or similar electronic device,that manipulates and transforms data represented as physical(electronic) quantities within the system's registers and memories intoother data similarly represented as physical quantities within thesystem's memories or registers or other such information storage,transmission or display devices.

The present disclosure can relate to an apparatus for performing one ormore of the operations described herein. This apparatus may be speciallyconstructed for the required purposes, or it may comprise a generalpurpose computer selectively activated or reconfigured by a computerprogram stored in the computer. Such a computer program may be stored ina machine (e.g. computer) readable storage medium, such as, but is notlimited to, any type of disk including floppy disks, optical disks,CD-ROMs, and magnetic-optical disks, read-only memories (ROMs), randomaccess memories (RAMs), erasable programmable ROMs (EPROMs),electrically erasable programmable ROMs (EEPROMs), flash memory,magnetic or optical cards, or any type of media suitable for storingelectronic instructions, and each coupled to a bus.

A machine-readable medium includes any mechanism for storing ortransmitting information in a form readable by a machine (e.g., acomputer). For example, a machine-readable medium includes read onlymemory (“ROM”); random access memory (“RAM”); magnetic disk storagemedia; optical storage media; flash memory devices; electrical, optical,acoustical or other form of propagated signals (e.g., carrier waves,infrared signals, digital signals, etc.); etc.

At least certain embodiments of the present disclosure include one orapplication programming interfaces in an environment with user interfacesoftware interacting with a software application. Various function callsor messages are transferred via the application programming interfacesbetween the user interface software and software applications.Transferring the function calls or messages may include issuing,initiating, invoking or receiving the function calls or messages.Example application programming interfaces transfer function calls toimplement scrolling, gesturing, and animating operations for a devicehaving a display region. An API may also implement functions havingparameters, variables, or pointers. An API may receive parameters asdisclosed or other combinations of parameters. In addition to the APIsdisclosed, other APIs individually or in combination can perform similarfunctionality as the disclosed APIs.

The display region is a form of a window. A window is a display regionwhich may not have a border and may be the entire display region or areaof a display. In some embodiments, a display region may have at leastone window and/or at least one view (e.g., web, text, or image content).A window may have at least one view. The methods, systems, andapparatuses disclosed can be implemented with display regions, windows,and/or views.

At least certain embodiments of the present disclosure include scrollingoperations for scrolling a display of a device. The scrolling operationsinclude bouncing a scrolled region in an opposite direction of a scrollwhen a scroll completes, rubberbanding a scrolled region by apredetermined maximum displacement when the scrolled region exceeds adisplay edge, and setting a scrolling angle that locks the scroll in ahorizontal or vertical direction.

At least certain embodiments of the present disclosure include gestureoperations for a display of a device. The gesture operations includeperforming a scaling transform such as a zoom in or zoom out in responseto a user input having two or more input points. The gesture operationsalso include performing a rotation transform to rotate an image or viewin response to a user input having two or more input points.

At least certain embodiments of the present disclosure include a methodfor performing animations for a display of a device. The method includesstarting at least one animation. The method further includes determiningthe progress of each animation. The method further includes completingeach animation based on a single timer. The single timer can be based ona redraw interval of the display hardware.

At least certain embodiments of the disclosure may be part of a digitalmedia player, such as a portable music and/or video media player, whichmay include a media processing system to present the media, a storagedevice to store the media and may further include a radio frequency (RF)transceiver (e.g., an RF transceiver for a cellular telephone) coupledwith an antenna system and the media processing system. In certainembodiments, media stored on a remote storage device may be transmittedto the media player through the RF transceiver. The media may be, forexample, one or more of music or other audio, still pictures, or motionpictures.

The portable media player may include a media selection device, such asa click wheel input device on an iPod® or iPod Nano® media player fromApple Computer, Inc. of Cupertino, Calif., a touch screen input device,pushbutton device, movable pointing input device or other input device.The media selection device may be used to select the media stored on thestorage device and/or the remote storage device. The portable mediaplayer may, in at least certain embodiments, include a display devicewhich is coupled to the media processing system to display titles orother indicators of media being selected through the input device andbeing presented, either through a speaker or earphone(s), or on thedisplay device, or on both display device and a speaker or earphone(s).In some embodiments, the display device and input device are integratedwhile in other embodiments the display device and input device areseparate devices. Examples of a portable media player are described inU.S. patent application Ser. No. 11/586,862, titled “Automated Responseto and Sensing of User Activity in Devices” and published U.S. patentapplication numbers 2003/0095096 and 2004/0224638 which have beenincorporated by reference into the present application.

Embodiments of the disclosure described herein may be part of othertypes of data processing systems, such as, for example, entertainmentsystems or personal digital assistants (PDAs), or general purposecomputer systems, or special purpose computer systems, or an embeddeddevice within another device, or cellular telephones which do notinclude media players, or multi touch tablet devices, or other multitouch devices, or devices which combine aspects or functions of thesedevices (e.g., a media player, such as an iPod®, combined with a PDA, anentertainment system, and a cellular telephone in one device). In thisdisclosure, electronic devices and consumer devices are types ofdevices.

FIG. 1 is flow chart of a method for responding to a user input of adevice. The method 100 includes receiving a user input at block 102. Theuser input may be in the form of an input key, button, wheel, touch, orother means for interacting with the device. The method 100 furtherincludes creating an event object in response to the user input at block104. The method 100 further includes determining whether the eventobject invokes a scroll or gesture operation at block 106. For example,a single touch that drags a distance across a display of the device maybe interpreted as a scroll operation. In one embodiment, a two or morefinger touch of the display may be interpreted as a gesture operation.In certain embodiments, determining whether the event object invokes ascroll or gesture operation is based on receiving a drag user input fora certain time period. The method 100 further includes issuing at leastone scroll or gesture call based on invoking the scroll or gestureoperation at block 108. The method 100 further includes responding to atleast one scroll call, if issued, by scrolling a window having a view(e.g., web, text, or image content) associated with the event objectbased on an amount of a scroll with the scroll stopped at apredetermined position in relation to the user input at block 110. Forexample, an input may end at a certain position on a display of thedevice. The scrolling may continue until reaching a predeterminedposition in relation to the last input received from the user. Themethod 100 further includes responding to at least one gesture call, ifissued, by changing a view associated with the event object based onreceiving a plurality of input points in the form of the user input atblock 112.

In certain embodiments of the present disclosure, scroll operationsinclude attaching scroll indicators to a content edge of a display.Alternatively, the scroll indicators can be attached to the displayedge. In some embodiments, user input in the form of a mouse/finger downcauses the scroll indicators to be displayed on the display edge,content edge, or window edge of the scrolled region. If a mouse/fingerup is then detected, the scroll indicators are faded out from thedisplay region, content edge, or window edge of the scrolled region.

In certain embodiments of the present disclosure, gesture operationsinclude responding to at least one gesture call, if issued, by rotatinga view associated with the event object based on receiving a pluralityof input points in the form of the user input. Gesture operations mayalso include scaling a view associated with the event object by zoomingin or zooming out based on receiving the user input.

In some embodiments, a device includes a display region having multipleviews or windows. Each window may have a multiple views includingsuperviews and subviews. It is necessary to determine which window,view, superview, or subview is contacted by a user input in the form ofa mouse up, mouse down, or drag, etc. An API can set various modes formaking this determination. In one embodiment, a pass mode sends mousedown, mouse up, and drag inputs to the nearest subview. In anotherembodiment, an intercept on drag mode sends a drag input to thesuperview while mouse up and down inputs are sent to the subview. Inanother embodiment, an intercept mode sends all drag, mouse up and downinputs to the superview. The superview may be scroller softwareoperating as a subclass of a view software. The subview may be viewsoftware operating as a subclass of the user interface software.

FIG. 2 illustrates details of an application programming interface inflow chart form according to certain teachings of the presentdisclosure. The application programming interface operates in anenvironment with user interface software interacting with a softwareapplication in order to provide a bounce operation. The method 200 forproviding a bounce operation includes transferring a set bounce call atblock 202. The method 200 further includes setting at least one ofmaximum and minimum bounce values at block 204. The minimum and maximumbounce values may be associated with at least one edge of a window thathas received a user input. The method 200 further includes causing abounce of a scrolled region in an opposite direction of a scroll basedon a region past the scrolled region being visible in a display regionat the end of the scroll at block 206. The scrolled region may be acontent region.

In certain embodiments of the present disclosure, transferring the setbounce call is either one of issuing, initiating, invoking or receivingthe set bounce call.

FIG. 3 illustrates details of an application programming interface inflow chart form according to certain teachings of the presentdisclosure. The application programming interface operates in anenvironment with user interface software interacting with a softwareapplication in order to provide a rubberband operation. The method 300for providing a rubberband operation includes transferring a rubberbandcall to cause rubberbanding a scrolled region displayed within a displayat block 302. The method 300 further includes transferring an edgerubberband call to set displacement values for at least one edge of thedisplay at block 304. In some embodiments, the displacement values areset for top and bottom edges, left and right edges, or all edges.

Rubberbanding a scrolled region according to the method 300 occurs by apredetermined maximum displacement value when the scrolled regionexceeds a display edge of a display of a device based on the scroll. Ifa user scrolls content of the display making a region past the edge ofthe content visible in the display, then the displacement value limitsthe maximum amount for the region outside the content. At the end of thescroll, the content slides back making the region outside of the contentno longer visible on the display.

In certain embodiments of the present disclosure, transferring therubberband call is either one of issuing, initiating, invoking orreceiving the rubberband call.

FIG. 4 is a schematic diagram illustrating an embodiment of userinterface of a portable electronic device 400 having a touch-sensitivedisplay 408. The display 408 may include a window 410. The window 410may include one or more displayed objects, such as information objects412-1 to 412-4. In an exemplary embodiment, the information objects 412may correspond to contact information for one or more individuals in alist of items. The displayed objects may be moved in response todetecting or determining movement 414 of a point of contact with thedisplay, such as that associated with one or more digits 416 of a user(which are not drawn to scale in FIG. 4). In some embodiments, movementof the displayed objects may be accelerated in response to detecting ordetermining accelerated movement of the point of contact. Whileembodiment 400 includes one window 410, in other embodiments there maybe two or more display windows. In addition, while embodiment 400illustrates movement 414 in a particular direction, in other embodimentsmovement of the displayed objects may be in response to movement 414 inone or more other directions, or in response to a scalar (i.e., adetermined or detected movement independent of the direction).

FIGS. 5A-5C illustrate the scrolling of a list of items on a device to aterminus of the list, at which point one or more displayed items at theend of the list smoothly bounce off the end of the display, reversedirection, and then optionally come to a stop. FIG. 5A is a schematicdiagram illustrating an embodiment of user interface of a portableelectronic device 400 having a touch-sensitive display. One or moredisplayed objects, such as information object 412-1 may be a distance512-1 from a terminus 514 of the list of items which is an edge of ascrolled region and may be moving with a velocity 510-1 while the listis being scrolled. Note that the terminus 514 is a virtual boundaryassociated with the displayed objects, as opposed to a physical boundaryassociated with the window 410 and/or the display 408. As illustrated inFIG. 5B, when the one or more displayed objects, such as the informationobject 412-1, reach or intersect with the terminus 514, the movementcorresponding to the scrolling may stop, i.e., the scrolling velocitymay be zero at an instant in time. As illustrated in FIG. 5C, the one ormore displayed objects, such as the information 412-1, may subsequentlyreverse direction. At a time after the intersection with the terminus514, the information object 412-1 may have velocity 510-2 and may be adistance 512-2 from the terminus 514. In some embodiments, the magnitudeof velocity 510-2 may be less than the magnitude of velocity 510-1 whenthe distance 512-2 equals the distance 512-1, i.e., the motion of theone or more displayed objects is damped after the scrolling list reachesand “bounces” at its terminus.

In at least some embodiments of the present disclosure, the method 200performs the bounce operations described in FIGS. 5A-5C. The bounce calltransferred at block 202 determines whether a bounce operation isenabled. The maximum and minimum bounces values determine the amount ofbouncing of the scrolled region in an opposite direction of the scroll.

FIGS. 6A-6D illustrate the scrolling of a list of items to a terminus ofthe list, at which point an area beyond the terminus is displayed andthe list is then scrolled in an opposite direction until the area beyondthe terminus is no longer displayed, in accordance with someembodiments. The rubberband operation of method 300 is illustrated inthe example of FIGS. 6A-6D with the listed items being email messages.FIGS. 6A-6D illustrate an exemplary user interface 3500A for managing aninbox in accordance with some embodiments. An analogous user interfacemay be used to display and manage other mailboxes (e.g., drafts, sent,trash, personal, etc.). In addition, other types of lists are possible,including but not limited to lists of instant message conversations,favorite phone numbers, contact information, labels, email folders,email addresses, physical addresses, ringtones, or album names.

If the list of emails fills more than the allotted screen area, the usermay scroll through the emails using vertically upward and/or verticallydownward swipe gestures on the touch screen. In the example of FIG. 6A,a portion of a list of emails is displayed in the screen area, includinga top displayed email 3530 from Bruce Walker and a bottom displayedemail 3532 from Kim Brook. A user performs a vertically downward swipegesture 3514 to scroll toward the top of the list. The verticallydownward gesture 3514 need not be exactly vertical; a substantiallyvertical gesture is sufficient. In some embodiments, a gesture within apredetermined angle of being perfectly vertical results in verticalscrolling.

As a result of detecting the vertically downward gesture 3514, in FIG.6B the displayed emails have shifted down, such that the previous bottomdisplayed email 3532 from Kim Brook is no longer displayed, the previoustop displayed email 3530 from Bruce Walker is now second from the top,and the email 3534 from Aaron Jones, which was not displayed in FIG. 6A,is now displayed at the top of the list.

In this example, the email 3534 from Aaron Jones is the first email inthe list and thus is the terminus of the list. Upon reaching this email3534, in response to continued detection of the vertically downwardgesture 3514, an area 3536 (FIG. 6C) above the first email 3534 (i.e.,beyond the terminus of the list) is displayed. In some embodiments, thearea displayed beyond the terminus of the list is visually indistinctfrom the background of the list. In FIG. 6C, both the area 3536 and thebackground of the emails (e.g., emails 3534 and 3530) are white and thusare visually indistinct.

Once vertically downward gesture 3514 is complete, such that acorresponding object is no longer detected on or near the touch screendisplay, the list is scrolled in an opposite direction until the area3536 is no longer displayed. FIG. 6D illustrates the result of thisscrolling in the opposite direction, the email 3534 from Aaron Jones isnow displayed at the top of the screen area allotted to the list and thearea 3536 is not displayed.

In the example of FIGS. 6A-6D, a vertically downward gesture resulted indisplay of an area beyond the first item in the list. As described inFIG. 3, the values for the predetermined maximum displacement (e.g.,display of an area beyond the first item in the list) are set at block304 for top and bottom edges or at block 306 for all edges of thewindow.

Similarly, a vertically upward gesture may result in display of an areabeyond the last item of the list, if the vertically upward gesturecontinues once the list has been scrolled to the last item. The lastitem may be considered a terminus of the list, similar to the firstitem. As discussed above, the gesture need not be exactly vertical toresult in vertical scrolling; a gesture within a predefined range ofangles from perfectly vertical is sufficient.

FIG. 7 illustrates details of an application programming interface inflow chart form according to certain teachings of the presentdisclosure. The application programming interface operates in anenvironment with user interface software interacting with a softwareapplication in order to provide a directional scrolling operation. Themethod 700 for operating through an application programming interface(API) includes transferring a directional scroll angle call to determineif directional scrolling is enabled at block 702. The method 700 furtherincludes transferring a directional scroll angle call to set a scrollangle for locking the scrolling in at least one of a vertical or ahorizontal direction at block 704. The method 700 further includeslocking the scrolling in the horizontal direction if a user input formsan angle with a horizontal direction that is less than or equal to afirst scroll angle at block 706. The method 700 further includes lockingthe scrolling in the vertical direction if a user input forms an anglewith the vertical direction that is less than or equal to a secondscroll angle at block 708.

In certain embodiments, a user input in the form of a drag forms anangle with the horizontal direction that is less than the first scrollangle. In this case, the user presumably intends to scroll in thehorizontal direction. The scrolling will be locked in the horizontaldirection until the user input exceeds the first scroll angle. A secondscroll angle may be used for locking the user input in the verticaldirection. The second scroll angle may be set equal to the first scrollangle.

FIG. 8 illustrates first and second scroll angles for locking a scrollof a display of a device in a horizontal or vertical direction. Thehorizontal direction 802 and vertical direction 804 are in reference toa window or a display of a device. As discussed in the method 700, auser input such as a drag movement forming an angle with the horizontaldirection 802 less than or equal to the first scrolling angle 806 or 808will lock the user input in the horizontal direction. In a similarmanner, a user input forming an angle with the vertical direction 810less than or equal to the second scrolling angle 810 or 812 will lockthe user input in the vertical direction. The first and second scrollingangles may be set at the same angle or at different angles as well. Forexample, the first and second scrolling angles may be set at 25 degrees.A user input less than or equal to 25 degrees with respect to thehorizontal or vertical direction will lock the scrolling in theappropriate direction.

In some embodiments, the horizontal and vertical locking angles can bedetermined in part by the aspect of the content. For example, content inthe form of a tall page may receive a larger vertical locking anglecompared to the horizontal locking angle.

FIG. 9 illustrates details of an application programming interface inflow chart form according to certain teachings of the presentdisclosure. The application programming interface operates in anenvironment with user interface software interacting with a softwareapplication in order to provide a deceleration scroll operation. Themethod 900 for providing the deceleration scroll operation includestransferring a deceleration scroll call to set a deceleration factor fora drag user input at block 902. The method 900 further includes slowingthe scroll to a stop based on the speed of the drag user input and thedeceleration factor at block 904.

In certain embodiments, a user input in the form of a drag invokes ascroll operation for a certain time period. The user input has a certainspeed. The scroll of the scrolled region of a window or a display regionof a display of a device will be stopped after the user input stops byapplying a deceleration factor to the speed of the user input during thedrag movement.

FIG. 10 illustrates details of an application programming interface inflow chart form according to certain teachings of the presentdisclosure. The application programming interface operates in anenvironment with user interface software interacting with a softwareapplication in order to provide a scroll hysteresis operation. Themethod 1000 for providing the scroll hysteresis operation includestransferring a scroll hysteresis call to determine whether a user inputinvokes a scroll at block 1002. The method 1000 further includes settinga hysteresis value for determining whether a user input invokes a scrollat block 1004.

In certain embodiments, a user input in the form of a drag over acertain distance across a display or window within a display of a deviceinvokes a scroll operation. The hysteresis value determines the certaindistance which the user input must drag across the display or windowprior to invoking a scroll operation. A user input that does not dragthe certain predetermined distance will not invoke a scroll operationand may be considered a mouse up or down input or other type of input.

FIG. 11 illustrates details of an application programming interface inflow chart form according to certain teachings of the presentdisclosure. The application programming interface operates in anenvironment with user interface software interacting with a softwareapplication in order to attach a scroll indicator to a scroll regionedge or a window edge of a device. In some embodiments, the scrollregion edge is associated with a content edge. The window or displayedge may be associated with the edge of a display region. The method1100 for providing the scroll indicator includes transferring a scrollindicator call to determine whether at least one scroll indicatorattaches to an edge of a scroll region or a window edge at block 1102. Ascroll indicator may be displayed on any display edge, window edge orscroll region edge. The method 1100 further includes optionallyattaching at least one scroll indicator to the edge of the scroll regionbased on the scroll indicator call at block 1104. Alternatively, themethod 1100 further includes optionally attaching at least one scrollindicator to the window edge of the view based on the scroll indicatorcall at block 1106.

In some embodiments, the operations of method 1100 can be altered,modified, combined, or deleted. For example, block 1104 can be deleted.Likewise, block 1106 can be deleted from the method 1100. Alternatively,the order of block 1104 and block 1106 can be switched. Other methodshaving various operations that have been disclosed within the presentdisclosure can also be altered, modified, rearranged, collapsed,combined, or deleted.

In certain embodiments of the present disclosure, transferring thescroll indicator call is either one of issuing, initiating, invoking orreceiving the scroll indicator call. For example, the user interfacesoftware (e.g., software kit or library) may receive the scrollindicator call from the software application.

FIG. 12 illustrates details of an application programming interface inflow chart form according to certain teachings of the presentdisclosure. The application programming interface operates in anenvironment with user interface software interacting with a softwareapplication in order to determine if an inadvertent user input contactsa view of a display of a device. The method 1200 includes transferringan inadvertent user input call to determine whether the user input wasinadvertent at block 1202. The method 1200 further includes ignoring theinadvertent user input based on the determination of the inadvertentuser input call at block 1204. In one embodiment, the inadvertent userinput call comprises a thumb detection call to determine whether theuser input was an inadvertent thumb.

In certain embodiments of the present disclosure, transferring theinadvertent user input call is either one of issuing, initiating,invoking or receiving the inadvertent user input call.

A gesture API provides an interface between an application and usersoftware in order to handle gesturing. Gesturing may include scaling,rotating, or other changes to a view, window, or display. A mask maymerely permit certain changes while limiting or not permitting otherchanges. Events of all kinds come into the application via a graphicsframework. They are enqueued, collaleced if necessary and dispatched. Ifthe events are system level events (e.g., application should suspend,device orientation has changed, etc) they are routed to the applicationhaving an instance of a class of the user interface software. If theevents are hand events based on a user input, the events are routed tothe window they occurred over. The window then routes these events tothe appropriate control by calling the instance's mouse and gesturemethods. The control that receives a mouse down or mouse enteredfunction will continue to get all future calls until the hand is lifted.If a second finger is detected, the gesture methods or functions areinvoked. These functions may include start, change, and end gesturecalls. The control that receives start gesture call will be sent allfuture change gesture calls until the gesture ends.

FIG. 13 illustrates details of an application programming interface inflow chart form according to certain teachings of the presentdisclosure. The application programming interface operates in anenvironment with user interface software interacting with a softwareapplication in order to provide a gesture operation. The method 1300 forproviding the gesture operation includes transferring a handle gestureevent call at block 1302. The method 1300 further includes transferringa gesture change call in response to the handle gesture event call atblock 1304.

In certain embodiments, a user input in the form of two or more pointsis received by a display of a device. A multi-touch driver of the devicereceives the user input and packages the event into an event object. Awindow server receives the event object and determines whether the eventobject is a gesture event object. If the window server determines that agesture event object has been received, then user interface softwareissues or transfers the handle gesture call at block 1302 to a softwareapplication associated with the view. The software application confirmsthat a gesture event has been received and passes the handle gesturecall to a library of the user interface software. The window server alsoassociates the gesture event object with the view that received the userinput. The library responds by transferring a gesture change call inresponse to the handle gesture event call at block 1304.

In one embodiment, a window or view associated with the user inputreceives the change call in order to perform the gesture event. The usersoftware that provides the view receives a gesture start event call, agesture changed event call, a zoom to scale setting for the view, and agesture end call. The gesture calls receive an input of a gesture eventwhich may be base event having a type such as a hand event, keyboardevent, system event, etc. A delegate associated with the applicationreceives a start gesture call, gesture did change call, and gesture didfinish call. The user software is dynamically linking into theapplication during the run time of the gesture process.

In some embodiments, the gesture changed function call contains thefollowing information about the gesture:

the number of fingers currently down;

the number of fingers initially down;

the rotation of the hand;

the scale of the hand;

the translation of the hand;

the position of the inner and outermost fingers; and

the pressure of the first finger.

In other embodiments, more information about each finger down may beincluded as follows.

the stage of the finger (just touch down, fully pressed, lifting off,etc);

the position of the finger;

the proximity of the finger (how hard you're touching);

the orientation of the finger (what angle the ovoid is at);

the length of the major and minor axis;

the velocity of the finger; and

the eccentricity of the finger's ovoid.

A gesture event object may be a chord event object having a chord count(e.g., number of fingers contacted the view or display), a chord startevent, a chord change event, and a chord end event. A chord change eventmay include a scaling or rotation transform.

FIG. 14 illustrates details of an application programming interface inflow chart form according to certain teachings of the presentdisclosure. The application programming interface operates in anenvironment with user interface software interacting with a softwareapplication in order to provide a scaling transform of a display region,window, or view of a display of a device. The method 1400 for providingthe scaling transform includes transferring a scaling transform call todetermine a scaling transform for a view associated with a user inputhaving a plurality of input points at block 1402. The method 1400further includes transferring a scaling gesture start call at block1404. The method 1400 further includes transferring a scaling gestureprogress call at block 1406. The method 1200 further includestransferring a scaling gesture end call at block 1408.

In certain embodiments, a user input in the form of two or more inputpoints (e.g., fingers) moves together or apart to invoke a gesture eventthat performs a scaling transform on the view associated with the userinput. A scale transform includes a minimum and maximum scale factor.FIG. 15 illustrates a display 1502 of a device having a scalingtransform of a view. The view 1504 (e.g., web, text, or image content)has a first scale factor. A user input (e.g., two fingers moving apart)associated with the view 1504 is interpreted as a gesture event to zoomout from view 1504 to view 1508 having a second scale factor thatexceeds the maximum scale factor of the view 1516. A snapback flagdetermines whether the zoom out can proceed past the maximum scalefactor to view 1508 prior to snapping back to the maximum scale factorassociated with view 1516.

FIG. 16A illustrates a display 1604 of a device having a first scalingfactor of a view 1616. A user input (e.g., two fingers 1608 and 1610moving together) associated with the view 1614 is interpreted as agesture event to zoom in from view 1614 to view 1664 having a secondscale factor as illustrated in FIG. 16B. The dashed regions 1602 and1650 represent the total area of the content with the only content beingdisplayed in the display area 1604 and 1652. In performing the scalingtransform from FIG. 16A to FIG. 16B, the center of the gesture event,center 1612 for FIG. 16A and center 1660 for FIG. 16B, remains in thesame position with respect to the display 1604. The scroll indicator1606 shrinks to become scroll indicator 1654 during the transform toindicate that a smaller portion of the total content 1650 is beingdisplayed on display 1604 as a result of the zoom in operation. Thedashed region 1650 is larger than the dashed region 1602 to representthat a larger portion of content is not being displayed on display 1652in FIG. 16B as a result of the zoom in operation.

In at least some embodiments of the present disclosure, a user desiresto change a view 1670 from a scale factor of 2× to a scale factor of 1×as illustrated in FIG. 16C. A first set of user inputs 1672 and 1674that move to the second set of user inputs 1676 and 1678 will decreasethe scale factor from 2× to 1×. It may be desirable for the user toscale from 2× to 1× without having to move the user inputs a largedistance across the view 1670. In an environment with user interfacesoftware interacting with a software application, a gesture scalingtransform flag may be set in order to determine a scaling transform fora view associated with a user input having a plurality of input points.The scaling transform flag scales either from a current scale factor toa minimum scale factor or from the current scale factor to a maximumscale factor. For example, a flag may be set at the position associatedwith a 1.5× scale factor and a third set of user inputs 1680 and 1682. Auser desiring to change the scale factor from 2× to 1× would only haveto move his fingers, the user inputs, from the first set 1672 and 1674to the third set 1680 and 1682 if the gesture scaling transform flag hasbeen set at a scale factor of 1.5×.

FIG. 17 illustrates details of an application programming interface inflow chart form according to certain teachings of the presentdisclosure. The application programming interface operates in anenvironment with user interface software interacting with a softwareapplication in order to provide a rotation transform of a view, window,or display region of a display of a device. The method 1700 forproviding the rotation transform includes transferring a rotationtransform call to determine a rotation transform for a view associatedwith a user input having a plurality of input points at block 1702. Themethod 1700 further includes transferring a rotation gesture start callat block 1704. The method 1700 further includes transferring a rotationgesture progress call at block 1706. The method 1700 further includestransferring a rotation gesture end call at block 1708.

In certain embodiments, a user input in the form of two or more inputpoints rotates to invoke a gesture event that performs a rotationtransform on the view associated with the user input. The rotationtransform includes a minimum and maximum degree of rotation forassociated minimum and maximum rotation views. The user input maytemporarily rotate a view past a maximum degree of rotation prior to theview snapping back to the maximum degree of rotation.

FIG. 18 illustrates details of an application programming interface inflow chart form according to certain teachings of the presentdisclosure. The application programming interface operates in anenvironment with user interface software interacting with a softwareapplication in order to notify a delegate of at least one animationassociated with a display region, window, or view of a display of adevice. A delay in the animation may be specified by the API. Also,multiple animations may be assigned priority by the API. The method 1800for notifying the delegate includes determining whether any animationoccurs at block 1802. The method 1800 further includes checking theprogress of an animation at block 1804. If progress has occurred, thenthe next state (e.g., position, opacity, or transform) of the animationcan be calculated at block 1806. If progress has completed at block1806, then at block 1808 it is determined whether the view associatedwith the completed animation is associated with a delegate. If so, adelegate call is transferred to notify the delegate of the animation forthe view at block 1810. The delegate operating under the control of thesoftware application can change other views in response to the viewbeing modified by the animation.

In certain embodiments, software invokes an animation that performs ascaling transform on the view associated with the user input. A displaymay include numerous views. The view being increased in size by thescaling transform may obstruct other views in which case the other viewsmay need to be reduced in size. Alternatively, the view being decreasedin size by the scaling transform may create additional area for otherviews to increase in size.

FIG. 19 is flow chart of a method for animating a display region,windows, or views displayed on a display of a device. The method 1900includes starting at least two animations at block 1902. The method 1900further includes determining the progress of each animation at block1904. The method 1900 further includes completing each animation basedon a single timer at block 1906.

In certain embodiments of the present disclosure, the single timerincludes a timer based on a redraw interval which is a time periodbetween the display of a current frame and a next frame of the displayof the device. In this case, changes in animation are updated to thedisplay during the redraw interval in order to display the changesduring the next frame of the display. The progress of each animation maybe calculated periodically or based upon a progress call.

The method 1900 may further include determining whether each animationis associated with a delegate. The delegate is then notified of theanimation. Other views not associated with an animation may be changeddepending on the software application controlling the delegate.

FIG. 20 is flow chart of a method for animating a display region,windows, or views displayed on a display of a device. The method 2000includes providing a single animation timer at block 2002. The method2000 further includes animating a plurality of animations with thesingle animation timer at block 2004. For example, a single timer maycontrol all animations which occur simultaneously. The animations mayinclude a transform, a frame, and an opacity animation. A animationtransform may include a scaling or rotation transform. A frame animationmay include resizing of a frame. An opacity animation changes theopacity from opaque to transparent or vice versa.

FIG. 21 illustrates details of an application programming interface inflow chart form according to certain teachings of the presentdisclosure. The application programming interface operates in anenvironment with user interface software interacting with multiplesoftware applications or processes in order to synchronize animationsassociated with multiple views or windows of a display of a device. Themethod 2100 for synchronizing the animations includes setting attributesof views independently with each view being associated with a process atblock 2102. For example, an attribute or property of a view may includea position, size, opacity, etc. An animation alters one or moreattributes from a first state to a second state. The method 2100 furtherincludes transferring a synchronization call to synchronize animationsfor the multiple views of the display at block 2104. The synchronizationcall may include input parameters or arguments such as an identificationof the synchronization of the processes and a list of the processes thatare requesting animation of the multiple views. In one embodiment, thesynchronization call includes the identification and the number ofprocesses that are requesting animation. In one embodiment, eachapplication or process sends a synchronization call at different times.The method 2100 further includes transferring a synchronizationconfirmation message when a synchronization flag is enabled at block2106. The synchronization flag can be enabled when the processes to besynchronized have each sent messages to a window server operating theuser interface software. The method 2100 further includes updating theattributes of the views from a first state to a second stateindependently at block 2108. In one embodiment, the window serverreceives the updated attributes from each process at different times.The method 2100 further includes transferring a start animation call todraw the requested animations when both processes have updatedattributes associated with the second state at block 2110.

In some embodiments, a first data structure or layer tree represents ahierarchy of layers that correspond to the views or windows of theprocesses. A second data structure or render tree represents a similarcopy of the layer tree. However, the render tree is not updated untilthe independent processes have completed their separate animations. Atthis time, the render tree updates and redraws the screen with the newanimations.

FIGS. 22A and 22B illustrate synchronizing the resizing of views orwindows of a display of a device. For example, a window 2210 associatedwith a first process with a size attribute may increase in size bychanging from a first state, window 2210 in FIG. 22A, to a second state,window 2210 in FIG. 22B. At approximately the same time, a second window2220 may decrease in size in proportion to the increase in size of thefirst window 2210. The method 2100 provides synchronization of theresizing of the windows 2210 and 2220 illustrated in FIGS. 22A and 22B.The animations in changing from the first state to the second state mayoccur incrementally and occur with the synchronization of method 2100.

FIG. 23 illustrates a method for switching ownership of a view of anapplication displayed on a display of a data processing device. Themethod 2300 includes constructing a data structure having a hierarchy oflayers with a layer being associated with a view and owning the view atblock 2302. The layers may be content, windows, video, images, text,media, or any other type of object for user interface of theapplication. The method 2300 further includes removing the layer fromthe data structure at block 2304. The method 2300 further includesswitching ownership of the view from the layer to the view at block2306.

In some embodiments, each layer from the data structure is associatedwith a view. The layer associated with the view sends a delegatefunction call to the view in order to generate content provided by theview. A first pointer reference points from the layer to the view. Asecond pointer reference points from the view to the layer. The numberof references pointing to an object such as the view is defined as theretained count of the object. The view may receive notification that thelayer will be removed from the data structure. Removing the layer fromthe data structure may occur based on the view associated with the layerbeing removed from the display of the device. When the layer is removedfrom the data structure or layer tree the pointer from the layer to theview will be removed. The view will have a retained count of zero and bedeallocated or removed from memory if the ownership of the view is notreversed. The view will have a retained count of at least one ifownership is reversed.

FIG. 24 illustrates a method for memory management of a view of anapplication displayed on a display of a device. The method 2400 includesconstructing a data structure having a hierarchy of layers with at leastone layer being associated with the view at block 2402. The method 2400further includes storing the data structure in memory at block 2404. Themethod 2400 further includes maintaining a retained count of the numberof references to the view from other objects at block 2406. The method2400 further includes deallocating the view from memory if the retainedcount is zero at block 2408. As discussed above, the retained count ofthe view will be decremented if the layer is removed from the datastructure. Removing the layer from the data structure may occur based onthe view associated with the layer being removed from the display of thedevice.

FIGS. 25A and 25B illustrate a data structure having a hierarchy oflayers with a layer being associated with a view. The data structureincludes layers 2502, 2504, and 2506. Layer 2506 is associated with theview 2510. The layer 2506 associated with the 2510 view sends a delegatecall to the view in order to generate content provided by the view. Afirst pointer reference 2508 points from the layer 2506 to the view2510. A second pointer reference 2512 points from the view 2510 to thelayer 2506. A third pointer reference 2532 may point from user interface(UI) controller 2530 to the view 2510. The UI controller 2530 maycontrol operations associated with the view 2510 such as scrolling theview 2510 in response to a user input. The view 2510 in FIG. 25A has aretained count of two based on the pointer references 2508 and 2532.

If the layer 2506 is removed from the data structure as illustrated inFIG. 25B, then the pointer 2508 is removed. View 2510 will have a lowerretained count as illustrated in FIG. 25B. If view 2510 has a retainedcount of zero, then the memory storing the view 2510 will bedeallocated.

FIG. 26 illustrates a method for compositing media and non-media contentof user interface for display on a device. The method 2600 includesconstructing a data structure having a hierarchy of layers associatedwith the user interface of the device at block 2602. The method 2600further includes determining whether each layer of the data structure isassociated with media or non-media content at block 2604. The datastructure or layer tree is traversed in order to determine whether eachof the layers of the data structure is associated with media ornon-media content. The method 2600 further includes detaching a layerassociated with media content from the data structure at block 2606. Themethod 2600 further includes storing media content in a first memorylocation at block 2606. The method 2600 further includes storingnon-media content in a second memory location at block 2608. The method2600 further includes compositing the media and non-media content fordisplay on the device at block 2610.

In some embodiments, compositing the media and non-media contentincludes retrieving the media content from the first memory location,retrieving the non-media content from the second memory location, andscanning the media and non-media content directly to the display. Thememory location can be any type of memory located in cache, main memory,a graphics processing unit, or other location within a device. The mediacontent may include video, video plug-in, audio, image, or other timevarying media. The media content may be in the form of a YUV model withthe Y representing a luminance component (the brightness) and U and Vrepresenting chrominance (color) components. The media content may bescanned to the display at a rate of substantially twenty to forty framesper second. The media content may be scaled prior to being scanned tothe display of the device.

The non-media content may include content, views, and images that do notrequire frequent updating. The non-media content may be in the form of aRGB model which is an additive model in which red, green, and blue(often used in additive light models) are combined in various ways toreproduce other colors. The non-media content may be scanned to thedisplay at a slower rate compared to the media content.

FIG. 27 illustrates a data structure or layer tree having a hierarchy oflayers. The layers can be associated with media and non-media content.For example, layer 2704 is associated with media content 2706 such as avideo. Layer 2710 is associated with non-media content 2712 which may beuser interface view for the video. Layers 2720, 2730, and 2740 areassociated with non-media content 2722, 2732, and 2742, respectively,that forms the components of the non-media content 2712. The method 2600will determine whether each layer of the data structure is associatedwith media or non-content. Any layers associated with media content suchas layer 2704 will be removed from the data structure and processed in aseparate memory location.

In some embodiments, the methods, systems, and apparatuses of thepresent disclosure can be implemented in various devices includingelectronic devices, consumer devices, data processing devices, desktopcomputers, portable computers, wireless devices, cellular devices,tablet devices, handheld devices, multi touch devices, multi touch dataprocessing devices, any combination of these devices, or other likedevices. FIGS. 4-6 and 28-33 illustrate examples of a few of thesedevices.

FIG. 28 illustrates a device 2800 according to one embodiment of thedisclosure. FIG. 28 shows a wireless device in a telephone configurationhaving a “candy-bar” style. In FIG. 28, the wireless device 2800 mayinclude a housing 2832, a display device 2834, an input device 2836which may be an alphanumeric keypad, a speaker 2838, a microphone 2840and an antenna 2842. The wireless device 2800 also may include aproximity sensor 2844 and an accelerometer 2846. It will be appreciatedthat the embodiment of FIG. 28 may use more or fewer sensors and mayhave a different form factor from the form factor shown in FIG. 28.

The display device 2834 is shown positioned at an upper portion of thehousing 2832, and the input device 2836 is shown positioned at a lowerportion of the housing 2832. The antenna 2842 is shown extending fromthe housing 2832 at an upper portion of the housing 2832. The speaker2838 is also shown at an upper portion of the housing 2832 above thedisplay device 2834. The microphone 2840 is shown at a lower portion ofthe housing 2832, below the input device 3286. It will be appreciatedthat the speaker 2838 and microphone 2840 can be positioned at anylocation on the housing, but are typically positioned in accordance witha user's ear and mouth, respectively.

The display device 2834 may be, for example, a liquid crystal display(LCD) which does not include the ability to accept inputs or a touchinput screen which also includes an LCD. The input device 2836 mayinclude, for example, buttons, switches, dials, sliders, keys or keypad,navigation pad, touch pad, touch screen, and the like. Any well-knownspeaker, microphone and antenna can be used for speaker 2838, microphone2840 and antenna 2842, respectively.

The data acquired from the proximity sensor 2844 and the accelerometer2846 can be combined together, or used alone, to gather informationabout the user's activities. The data from the proximity sensor 2844,the accelerometer 2846 or both can be used, for example, toactivate/deactivate a display backlight, initiate commands, makeselections, control scrolling, gesturing, animating or other movement ina display, control input device settings, or to make other changes toone or more settings of the device. In certain embodiments of thepresent disclosure, the device 2800 can be used to implement at leastsome of the methods discussed in the present disclosure.

FIG. 29 shows a device 2950 in accordance with one embodiment of thedisclosure. The device 2950 may include a housing 2952, a display/inputdevice 2954, a speaker 2956, a microphone 2958 and an optional antenna2960 (which may be visible on the exterior of the housing or may beconcealed within the housing). The device 2950 also may include aproximity sensor 2962 and an accelerometer 2964. The device 2950 may bea cellular telephone or a device which is an integrated PDA and acellular telephone or a device which is an integrated media player and acellular telephone or a device which is both an entertainment system(e.g. for playing games) and a cellular telephone, or the device 2950may be other types of devices described herein. In one particularembodiment, the device 2950 may include a cellular telephone and a mediaplayer and a PDA, all contained within the housing 2952. The device 2950may have a form factor which is small enough that it fits within thehand of a normal adult and is light enough that it can be carried in onehand by an adult. It will be appreciated that the term “portable” meansthe device can be easily held in an adult user's hands (one or both);for example, a laptop computer and an iPod are portable devices.

In one embodiment, the display/input device 2954 may include amulti-point touch input screen in addition to being a display, such asan LCD. In one embodiment, the multi-point touch screen is a capacitivesensing medium configured to detect multiple touches (e.g., blobs on thedisplay from a user's face or multiple fingers concurrently touching ornearly touching the display) or near touches (e.g., blobs on thedisplay) that occur at the same time and at distinct locations in theplane of the touch panel and to produce distinct signals representativeof the location of the touches on the plane of the touch panel for eachof the multiple touches.

In certain embodiments of the present disclosure, the device 2800 can beused to implement at least some of the methods discussed in the presentdisclosure.

FIGS. 30A and 30B illustrate a device 3070 according to one embodimentof the disclosure. The device 3070 may be a cellular telephone whichincludes a hinge 3087 that couples a display housing 3089 to a keypadhousing 3091. The hinge 3087 allows a user to open and close thecellular telephone so that it can be placed in at least one of twodifferent configurations shown in FIGS. 30A and 30B. In one particularembodiment, the hinge 3087 may rotatably couple the display housing tothe keypad housing. In particular, a user can open the cellulartelephone to place it in the open configuration shown in FIG. 30A andcan close the cellular telephone to place it in the closed configurationshown in FIG. 30B. The keypad housing 3091 may include a keypad 3095which receives inputs (e.g. telephone number inputs or otheralphanumeric inputs) from a user and a microphone 3097 which receivesvoice input from the user. The display housing 3089 may include, on itsinterior surface, a display 3093 (e.g. an LCD) and a speaker 3098 and aproximity sensor 3084; on its exterior surface, the display housing 3089may include a speaker 3096, a temperature sensor 3094, a display 3088(e.g. another LCD), an ambient light sensor 3092, and a proximity sensor3084A. Hence, in this embodiment, the display housing 3089 may include afirst proximity sensor on its interior surface and a second proximitysensor on its exterior surface.

In at least certain embodiments, the device 3070 may contain componentswhich provide one or more of the functions of a wireless communicationdevice such as a cellular telephone, a media player, an entertainmentsystem, a PDA, or other types of devices described herein. In oneimplementation of an embodiment, the device 3070 may be a cellulartelephone integrated with a media player which plays MP3 files, such asMP3 music files.

Each of the devices shown in FIGS. 4, 5A, 5B, 5C, 6A, 6B, 6C, 6D, 28,29, 30A and 30B may be a wireless communication device, such as acellular telephone, and may include a plurality of components whichprovide a capability for wireless communication. FIG. 31 shows anembodiment of a wireless device 3070 which includes the capability forwireless communication. The wireless device 3070 may be included in anyone of the devices shown in FIGS. 4, 5A, 5B, 5C, 6A, 6B, 6C, 6D, 28, 29,30A and 30B, although alternative embodiments of those devices of FIGS.4, 5A, 5B, 5C, 6A, 6B, 6C, 6D, 28, 29, 30A and 30B may include more orfewer components than the Wireless device 3070.

Wireless device 3070 may include an antenna system 3101. Wireless device3070 may also include a digital and/or analog radio frequency (RF)transceiver 3102, coupled to the antenna system 3101, to transmit and/orreceive voice, digital data and/or media signals through antenna system3101.

Wireless device 3070 may also include a digital processing system 3103to control the digital RF transceiver and to manage the voice, digitaldata and/or media signals. Digital processing system 3103 may be ageneral purpose processing device, such as a microprocessor orcontroller for example. Digital processing system 3103 may also be aspecial purpose processing device, such as an ASIC (application specificintegrated circuit), FPGA (field-programmable gate array) or DSP(digital signal processor). Digital processing system 3103 may alsoinclude other devices, as are known in the art, to interface with othercomponents of wireless device 3070. For example, digital processingsystem 3103 may include analog-to-digital and digital-to-analogconverters to interface with other components of wireless device 3070.Digital processing system 3103 may include a media processing system3109, which may also include a general purpose or special purposeprocessing device to manage media, such as files of audio data.

Wireless device 3070 may also include a storage device 3104, coupled tothe digital processing system, to store data and/or operating programsfor the Wireless device 3070. Storage device 3104 may be, for example,any type of solid-state or magnetic memory device.

Wireless device 3070 may also include one or more input devices 3105,coupled to the digital processing system 3103, to accept user inputs(e.g., telephone numbers, names, addresses, media selections, etc.)Input device 3105 may be, for example, one or more of a keypad, atouchpad, a touch screen, a pointing device in combination with adisplay device or similar input device.

Wireless device 3070 may also include at least one display device 33106,coupled to the digital processing system 3103, to display informationsuch as messages, telephone call information, contact information,pictures, movies and/or titles or other indicators of media beingselected via the input device 3105. Display device 3106 may be, forexample, an LCD display device. In one embodiment, display device 3106and input device 3105 may be integrated together in the same device(e.g., a touch screen LCD such as a multi-touch input panel which isintegrated with a display device, such as an LCD display device). Thedisplay device 3106 may include a backlight 3106A to illuminate thedisplay device 3106 under certain circumstances. It will be appreciatedthat the Wireless device 3070 may include multiple displays.

Wireless device 3070 may also include a battery 3107 to supply operatingpower to components of the system including digital RF transceiver 3102,digital processing system 3103, storage device 3104, input device 3105,microphone 3105A, audio transducer 3108, media processing system 3109,sensor(s) 3110, and display device 3106. Battery 3107 may be, forexample, a rechargeable or non-rechargeable lithium or nickel metalhydride battery. Wireless device 3070 may also include audio transducers3108, which may include one or more speakers, and at least onemicrophone 3105A. In certain embodiments of the present disclosure, thewireless device 3070 can be used to implement at least some of themethods discussed in the present disclosure.

FIG. 32 shows another example of a device according to an embodiment ofthe disclosure. This device 3200 may include a processor, such asmicroprocessor 3202, and a memory 3204, which are coupled to each otherthrough a bus 3206. The device 3200 may optionally include a cache 3208which is coupled to the microprocessor 3202. This device may alsooptionally include a display controller and display device 3210 which iscoupled to the other components through the bus 3206. One or moreinput/output controllers 3212 are also coupled to the bus 3206 toprovide an interface for input/output devices 3214 and to provide aninterface for one or more sensors 3216 which are for sensing useractivity. The bus 3206 may include one or more buses connected to eachother through various bridges, controllers, and/or adapters as is wellknown in the art. The input/output devices 3214 may include a keypad orkeyboard or a cursor control device such as a touch input panel.Furthermore, the input/output devices 3214 may include a networkinterface which is either for a wired network or a wireless network(e.g. an RF transceiver). The sensors 3216 may be any one of the sensorsdescribed herein including, for example, a proximity sensor or anambient light sensor. In at least certain implementations of the device3200, the microprocessor 3202 may receive data from one or more sensors3216 and may perform the analysis of that data in the manner describedherein. For example, the data may be analyzed through an artificialintelligence process or in the other ways described herein. As a resultof that analysis, the microprocessor 3202 may then automatically causean adjustment in one or more settings of the device.

In certain embodiments of the present disclosure, the device 3200 can beused to implement at least some of the methods discussed in the presentdisclosure.

FIGS. 33A-C show another example of a device according to at leastcertain embodiments of the disclosure. FIG. 33A illustrates a laptopdevice 3300 with a keyboard 3302, a body 3304, a display frame 3306, anda display 3308. The laptop device 3300 can be converted into a tabletdevice as illustrated in FIG. 33B and FIG. 33C. FIG. 33B illustrates theconversion of the laptop device into a tablet device. An edge of adisplay frame 3356 containing a display 3358 is slide within the body3354 across the top of a keyboard 3352 until forming a tablet device asillustrated in FIG. 33C. The tablet device with a display 2362 and adisplay frame 3366 rests on top of a body 3360.

In certain embodiments of the present disclosure, the laptop device 3300can be used to implement at least some of the methods discussed in thepresent disclosure.

FIG. 34 illustrates details of an application programming interface inflow chart form according to certain teachings of the presentdisclosure. The application programming interface operates in anenvironment with user interface software interacting with a softwareapplication. In some embodiments, a hierarchy of views operates on topof a hierarchy of layers within the user interface software. The APIoperates as illustrated in method 3400 that includes constructing ahierarchy of views operating on top of a hierarchy of layers at block3402. The method 3400 further includes providing access to the hierarchyof views without providing access to the hierarchy of layers at block3404. An application may interact with the hierarchy of views via theAPI without accessing the hierarchy of layers operating below thehierarchy of views.

In some embodiments, a platform provides various scrolling, gesturing,and animating operations. The platform includes hardware components andan operating system. The hardware components may include a processingunit coupled to an input panel and a memory coupled to the processor.The operating system includes one or more programs that are stored inthe memory and configured to be executed by the processing unit. One ormore programs include various instructions for transferring functioncalls or messages through an application programming interface in orderto perform various scrolling, gesturing, and animating operations.

In an embodiment, the one or more programs include instructions fortransferring a scaling transform call through an API to determine ascaling transform for a view associated with a user input having aplurality of input points. In an embodiment, the one or more programsinclude instructions for transferring a rubberband call through an APIto determine a rotation transform for a view associated with a userinput having a plurality of input points.

In some embodiments, the platform includes a framework containing alibrary of software code. The framework interacts with the programs ofthe platform to provide application programming interfaces forperforming various scrolling, gesturing, and animating operations. Theframework also includes associated resources (e.g., images, text, etc.)that are stored in a single directory.

In an embodiment, the library of the framework provides an API forspecifying a scaling transform for a view associated with a user inputhaving a plurality of input points. In an embodiment, the library of theframework provides an API for specifying a scaling transform for a viewassociated with a user input having a plurality of input points.

In the foregoing specification, the disclosure has been described withreference to specific exemplary embodiments thereof. It will be evidentthat various modifications may be made thereto without departing fromthe broader spirit and scope of the disclosure as set forth in thefollowing claims. The specification and drawings are, accordingly, to beregarded in an illustrative sense rather than a restrictive sense.

The invention claimed is:
 1. A machine readable non-transitory mediumstoring executable program instructions, which, when executed, cause anelectronic device with a display to perform a method comprising:receiving a user input that comprises a plurality of input pointstouching the display, wherein the user input generates a gesture event;transferring a gesture start event function call between user interfacesoftware and a software application based on the user input, wherein thegesture start event function call includes a first list of two or moreinput points touching the display at a first time; and transferring agesture changed event function call between the user interface softwareand the software application based on the user input, wherein thegesture changed event function call includes a second list of two ormore input points touching the display at a second time.
 2. The mediumof claim 1, wherein at least one of the gesture start event functioncall and the gesture changed event function call further comprisesinformation for at least one of the following: a rotation of at leastone input point; a scaling of the input points; a translation of theinput points; a position of the inner and outermost input points; and apressure of a first input point.
 3. The medium of claim 1, wherein atleast one of the gesture start event function call and the gesturechanged event function call further comprises a stage of an input pointincluding just touch down, fully pressed, or lifting off.
 4. The mediumof claim 1, wherein the first list further comprises a position of eachinput point of the first list and a first number of input pointstouching the display, wherein the second list further comprises aposition of each input point of the second list and a second number ofinput points touching the display.
 5. The medium of claim 1, wherein atleast one of the gesture start event function call and the gesturechanged event function call further comprises a proximity of an inputpoint with respect to the display, which indicates a pressure of theinput point.
 6. The medium of claim 1, wherein at least one of thegesture start event function call and the gesture changed event functioncall further comprises an orientation corresponding to a single inputpoint of the plurality of input points touching the display.
 7. Themedium of claim 1, wherein at least one of the gesture start eventfunction call and the gesture changed event function call furthercomprises a length of a major axis or a minor axis of an input point. 8.The medium of claim 1, wherein at least one of the gesture start eventfunction call and the gesture changed event function call furthercomprises a velocity of an input point.
 9. The medium of claim 1,wherein transferring a gesture start event function call between userinterface software and a software application based on the user inputincludes transferring the gesture start event function call through anapplication programming interface between the user interface softwareand the software application based on the user input; transferring thegesture changed event function call between the user interface softwareand the software application includes transferring the gesture changedevent function call through the application programming interfacebetween the user interface software and the software application basedon the user input; and the method includes transferring a gesture endfunction call through the application programming interface between theuser interface software and the software application based on the userinput.
 10. A method, comprising: receiving a user input that comprises aplurality of input points touching a display of an electronic device,wherein the user input generates a gesture event; transferring a gesturestart event function call between user interface software and a softwareapplication based on the user input, wherein the gesture start eventfunction call includes a first list of two or more input points touchingthe display at a first time; and transferring a gesture changed eventfunction call between the user interface software and the softwareapplication based on the user input, wherein the gesture changed eventfunction call includes a second list of two or more input pointstouching the display at a second time.
 11. The method of claim 10,further comprising: transferring a gesture end function call based onthe gesture event.
 12. The method of claim 10, wherein at least one ofthe gesture start event function call and the gesture changed eventfunction call further comprises a stage of an input point including justtouch down, fully pressed, or lifting off.
 13. The method of claim 10,wherein: at least one of the gesture start event function call and thegesture changed event function call further comprises a proximity of aninput point with respect to the touch device, which indicates a pressureof the input point, at least the one of the gesture start event functioncall and the gesture changed event function call further comprises anorientation corresponding to a single input point of the plurality ofinput points touching the display, at least the one of the gesture startevent function call and the gesture changed event function call furthercomprises a length of a major axis or a minor axis of an input point,and at least the one of the gesture start event function call and thegesture changed event function call further comprises a velocity of aninput point.
 14. The method of claim 10, wherein transferring thegesture start event function call between the user interface softwareand the software application based on the user input includestransferring the gesture start event function call through anapplication programming interface between the user interface softwareand the software application based on the user input, and transferringthe gesture changed event function call between the user interfacesoftware and the software application based on the user input includestransferring the gesture changed event function call through theapplication programming interface between the user interface softwareand the software application based on the user input.
 15. A devicecomprising: an input panel which is configured to receive user inputs; adisplay device integrated with the input panel; one or more processorscoupled to the input panel; and a memory storing one or more programsconfigured to be executed by the one or more processors, the one or moreprograms including: instructions for receiving a user input thatcomprises a plurality of input points touching or near the input panel,wherein the user input generates a gesture event; instructions fortransferring a gesture start event function call between user interfacesoftware and a software application based on the user input, wherein thegesture start event function call includes a first list of two or moreinput points touching or near the input panel at a first time; andinstructions for transferring a gesture changed event function callbetween the user interface software and the software application basedon the user input, wherein the gesture changed event function callincludes a second list of two or more input points touching or near theinput panel at a second time.
 16. The device of claim 15, wherein theone or more programs further include: instructions for transferring agesture end function call based on the gesture event.
 17. The device ofclaim 16, wherein at least one of the gesture start event function calland the gesture changed event function call further comprises: a stageof an input point including just touch down, fully pressed, or liftingoff.
 18. The device of claim 16, wherein the first list furthercomprises a position of each input point of the first list and a firstnumber of input points touching the input panel, wherein the second listfurther comprises a position of each input point of the second list anda second number of input points touching the input panel.
 19. The deviceof claim 16, wherein at least one of the gesture start event functioncall and the gesture changed event function call further comprises aproximity of an input point with respect to the display device, whichindicates a pressure of the input point.
 20. The device of claim 16,wherein at least one of the gesture start event function call and thegesture changed event function call further comprises: an orientationcorresponding to a single input point of the plurality of input pointstouching or near the input panel, a length of a major axis or a minoraxis of an input point, and a velocity of an input point.
 21. The deviceof claim 15, wherein the instructions for transferring the gesture startevent function call between the user interface software and the softwareapplication based on the user input include instructions fortransferring the gesture start event function call through anapplication programming interface between the user interface softwareand the software application based on the user input, and theinstructions for transferring the gesture changed event function callbetween the user interface software and the software application basedon the user input include instructions for transferring the gesturechanged event function call through the application programminginterface between the user interface software and the softwareapplication based on the user input.
 22. A device comprising: an inputpanel which is configured to receive user inputs a display deviceintegrated with the input panel; one or more processors coupled to theinput panel; and a memory storing one or more programs configured to beexecuted by the one or more processors, the one or more programsincluding: instructions for receiving a user input that comprises aplurality of input points touching the input panel; instructions forgenerating an event object in response to the user input; instructionsfor transferring a gesture changed event function call between userinterface software and a software application in response to a change inthe user input; and instructions for transferring a gesture end functioncall between the user interface software and the software applicationwhen at least one input point, of the plurality of input points,associated with the user input is removed from the input panel.
 23. Thedevice of claim 22, wherein the one or more programs further include:instructions for generating a zoom to scale setting for a window or aview of the display in response to the change in the user input.
 24. Thedevice of claim 22, wherein the change in the user input comprises arotation to invoke a gesture event that performs a rotation transform ona view associated with the user input.
 25. The device of claim 24,wherein the rotation transform sets a start rotation angle and an endrotation angle.
 26. The device of claim 22, wherein the one or moreprograms include: instructions for transferring the gesture changedevent function call, through an application programming interfacebetween the user interface software and the software application, inresponse to the change in the user input; and instructions fortransferring the gesture end function call, through the applicationprogramming interface between the user interface software and thesoftware application, when the at least one input point, of theplurality of input points, associated with the user input is removedfrom the input panel.